A realistic evaluation : the case of protocol-based care

Background 'Protocol based care' was envisioned by policy makers as a mechanism for delivering on the service improvement agenda in England. Realistic evaluation is an increasingly popular approach, but few published examples exist, particularly in implementation research. To fill this gap, within this paper we describe the application of a realistic evaluation approach to the study of protocol-based care, whilst sharing findings of relevance about standardising care through the use of protocols, guidelines, and pathways. Methods Situated between positivism and relativism, realistic evaluation is concerned with the identification of underlying causal mechanisms, how they work, and under what conditions. Fundamentally it focuses attention on finding out what works, for whom, how, and in what circumstances. Results In this research, we were interested in understanding the relationships between the type and nature of particular approaches to protocol-based care (mechanisms), within different clinical settings (context), and what impacts this resulted in (outcomes). An evidence review using the principles of realist synthesis resulted in a number of propositions, i.e., context, mechanism, and outcome threads (CMOs). These propositions were then 'tested' through multiple case studies, using multiple methods including non-participant observation, interviews, and document analysis through an iterative analysis process. The initial propositions (conjectured CMOs) only partially corresponded to the findings that emerged during analysis. From the iterative analysis process of scrutinising mechanisms, context, and outcomes we were able to draw out some theoretically generalisable features about what works, for whom, how, and what circumstances in relation to the use of standardised care approaches (refined CMOs). Conclusions As one of the first studies to apply realistic evaluation in implementation research, it was a good fit, particularly given the growing emphasis on understanding how context influences evidence-based practice. The strengths and limitations of the approach are considered, including how to operationalise it and some of the challenges. This approach provided a useful interpretive framework with which to make sense of the multiple factors that were simultaneously at play and being observed through various data sources, and for developing explanatory theory about using standardised care approaches in practice

It’s More Complicated than That; Comment on “Translating Evidence into Healthcare Policy and Practice: Single Versus Multi-Faceted Implementation Strategies – Is There a Simple Answer to a Complex Question?”

In this commentary the findings from a systematic review that concluded there is no compelling evidence to suggest that implementing complicated, multi-faceted interventions is more effective than simple, single component interventions to changing healthcare professional’s behaviour are considered through the lens of Harvey and Kitson’s editorial. Whilst an appealing conclusion, it is one that hides a myriad of complexities. These include issues concerning how best to tailor interventions and how best to evaluate such efforts. These are complex issues that do not have simple solutions

A case study evaluation of implementation of a care pathway to support normal birth in one English birth centre: anticipated benefits and unintended consequences

Background: The policy drive for the UK National Health Service (NHS) has focused on the need for high quality services informed by evidence of best practice. The introduction of care pathways and protocols to standardise care and support implementation of evidence into practice has taken place across the NHS with limited evaluation of their impact. A multi-site case study evaluation was undertaken to assess the impact of use of care pathways and protocols on clinicians, service users and service delivery. One of the five sites was a midwifery-led Birth Centre, where an adapted version of the All Wales Clinical Pathway for Normal Birth had been implemented. Methods: The overarching framework was realistic evaluation. A case study design enabled the capture of data on use of the pathway in the clinical setting, use of multiple methods of data collection and opportunity to study and understand the experiences of clinicians and service users whose care was informed by the pathway. Women attending the Birth Centre were recruited at their 36 week antenatal visit. Episodes of care during labour were observed, following which the woman and the midwife who cared for her were interviewed about use of the pathway. Interviews were also held with other key stakeholders from the study site. Qualitative data were content analysed. Results: Observations were undertaken of four women during labour. Eighteen interviews were conducted with clinicians and women, including the women whose care was observed and the midwives who cared for them, senior midwifery managers and obstetricians. The implementation of the pathway resulted in a number of anticipated benefits, including increased midwifery confidence in skills to support normal birth and promotion of team working. There were also unintended consequences, including concerns about a lack of documentation of labour care and negative impact on working relationships with obstetric and other midwifery colleagues. Women were unaware their care was informed by a care pathway. Conclusion: Care pathways are complex interventions which generate a number of consequences for practice. Those considering introduction of pathways need to ensure all relevant stakeholders are engaged with this and develop robust evaluation strategies to accompany implementation

Making authentic: exploring boundary objects and bricolage in knowledge mobilisation through National Health Service-university partnerships

Background: In healthcare, bridging the research-to-practice gap is a top priority. Knowledge mobilisation scholars suggest that this gap can be closed through collaboration between knowledge users and producers. The concept of boundary objects – shared things and ideas that enable communication – has gained popularity across various collaborative work practices, but their potential within knowledge mobilisation in health care is understudied. An ongoing challenge for designers of boundary objects is how to create objects that are valued and shared both in principle and in practice. Aims and objectives: This paper reports on a study of boundary objects used during knowledge mobilisation through NHS-university partnerships called Collaborations for Leadership in Applied Health Research and Care (CLAHRCs). The distinction is investigated between boundary objects-in-theory and boundary objects-in-use, considering whether the latter possess specific characteristics which make them more effective during knowledge mobilisation. Methods: A qualitative case study of three CLAHRCs was conducted. Twenty-one people employed as ‘boundary spanners’ were interviewed to explore whether boundary objects played a role in knowledge mobilisation. Findings: The most effective boundary objects-in-use were co-produced through a process of bricolage. These possessed high levels of meaningfulness and resonance, and reconciled multiple user perspectives. Together these properties contributed to the overall authenticity of boundary objects-in-use. Discussion and conclusion: This paper helps to explain why designated boundary objects frequently fail in practice, and why there is a need to focus on understanding boundary objects based on symbolic, rather than structural, dimensions

Leadership for Evidence-Based Practice: Strategic and Functional Behaviors for Institutionalizing EBP

Making evidence-based practice (EBP) a reality throughout an organization is a challenging goal in healthcare services. Leadership has been recognized as a critical element in that process. However, little is known about the exact role and function of various levels of leadership in the successful institutionalization of EBP within an organization

Collaboration and co-production knowledge in healthcare: opportunities and challenges

Over time there has been a shift, at least in the rhetoric, from a pipeline conceptualisation of knowledge implementation, to one that recognises the potential of more collaboration, co-productive approaches to knowledge production and use. In this editorial, which is grounded in our research and collective experience, we highlight both the potential and challenge with collaboration and co-production. This includes issues about stakeholder engagement, governance arrangements, and capacity and capability for working in a coproductive way. Finally, we reflect on the fact that this approach is not a panacea, but is accompanied by some philosophical and practical challenges

Implementation, impact and costs of policies for safe staffing in acute NHS trusts

The aim of this research has been to describe the implementation of safe staffing policies in NHS general acute Trusts in England looking at costs and consequences, and examining the factors that have influenced implementation. A mix of qualitative and quantitative methods were used (national survey, analysis of national data, four case studies, realist evaluation) to examine the impact of policies nationally, and explore commonality and variation in local responses to safe staffing policies. The report is based on independent research commissioned and funded by the NIHR Policy Research Programme (“Implementation, Impact and Costs of Policies for Safe Staffing in Acute NHS Trusts”, PR-ST-1115-10017). The views expressed are those of the authors and not necessarily those of the NIHR, the Department of Health and Social Care, ‘arm’s length bodies’ or other government departments

The implausibility of ‘usual care’ in an open system: sedation and weaning practices in Paediatric Intensive Care Units (PICUs) in the United Kingdom (UK)

Background: The power of the randomised controlled trial depends upon its capacity to operate in a closed system whereby the intervention is the only causal force acting upon the experimental group and absent in the control group, permitting a valid assessment of intervention efficacy. Conversely, clinical arenas are open systems where factors relating to context, resources, interpretation and actions of individuals will affect implementation and effectiveness of interventions. Consequently, the comparator (usual care) can be difficult to define and variable in multi-centre trials. Hence outcomes cannot be understood without considering usual care and factors that may affect implementation and impact on the intervention. Methods: Using a fieldwork approach, we describe PICU context, ‘usual’ practice in sedation and weaning from mechanical ventilation, and factors affecting implementation prior to designing a trial involving a sedation and ventilation weaning intervention. We collected data from 23 UK PICUs between June and November 2014 using observation, individual and multi-disciplinary group interviews with staff. Results: Pain and sedation practices were broadly similar in terms of drug usage and assessment tools. Sedation protocols linking assessment to appropriate titration of sedatives and sedation holds were rarely used (9 % and 4 % of PICUs respectively). Ventilator weaning was primarily a medical-led process with 39 % of PICUs engaging senior nurses in the process: weaning protocols were rarely used (9 % of PICUs). Weaning methods were variably based on clinician preference. No formal criteria or use of spontaneous breathing trials were used to test weaning readiness. Seventeen PICUs (74 %) had prior engagement in multi-centre trials, but limited research nurse availability. Barriers to previous trial implementation were intervention complexity, lack of belief in the evidence and inadequate training. Facilitating factors were senior staff buy-in and dedicated research nurse provision. Conclusions: We examined and identified contextual and organisational factors that may impact on the implementation of our intervention. We found usual practice relating to sedation, analgesia and ventilator weaning broadly similar, yet distinctively different from our proposed intervention, providing assurance in our ability to evaluate intervention effects. The data will enable us to develop an implementation plan; considering these factors we can more fully understand their impact on study outcomes

A pragmatic cluster randomised trial evaluating three implementation interventions

Background Implementation research is concerned with bridging the gap between evidence and practice through the study of methods to promote the uptake of research into routine practice. Good quality evidence has been summarised into guideline recommendations to show that peri-operative fasting times could be considerably shorter than patients currently experience. The objective of this trial was to evaluate the effectiveness of three strategies for the implementation of recommendations about peri-operative fasting. Methods A pragmatic cluster randomised trial underpinned by the PARIHS framework was conducted during 2006 to 2009 with a national sample of UK hospitals using time series with mixed methods process evaluation and cost analysis. Hospitals were randomised to one of three interventions: standard dissemination (SD) of a guideline package, SD plus a web-based resource championed by an opinion leader, and SD plus plan-do-study-act (PDSA). The primary outcome was duration of fluid fast prior to induction of anaesthesia. Secondary outcomes included duration of food fast, patients' experiences, and stakeholders' experiences of implementation, including influences. ANOVA was used to test differences over time and interventions. Results Nineteen acute NHS hospitals participated. Across timepoints, 3,505 duration of fasting observations were recorded. No significant effect of the interventions was observed for either fluid or food fasting times. The effect size was 0.33 for the web-based intervention compared to SD alone for the change in fluid fasting and was 0.12 for PDSA compared to SD alone. The process evaluation showed different types of impact, including changes to practices, policies, and attitudes. A rich picture of the implementation challenges emerged, including inter-professional tensions and a lack of clarity for decision-making authority and responsibility. Conclusions This was a large, complex study and one of the first national randomised controlled trials conducted within acute care in implementation research. The evidence base for fasting practice was accepted by those participating in this study and the messages from it simple; however, implementation and practical challenges influenced the interventions' impact. A set of conditions for implementation emerges from the findings of this study, which are presented as theoretically transferable propositions that have international relevance. Trial registration ISRCTN18046709 - Peri-operative Implementation Study Evaluation (POISE

Accessibility and implementation in UK services of an effective depression relapse prevention programme - mindfulness-based cognitive therapy (MBCT): ASPIRE study protocol

notes: PMCID: PMC4036706types: Journal Article© 2014 Rycroft-Malone et al.; licensee BioMed Central Ltd.Mindfulness-based cognitive therapy (MBCT) is a cost-effective psychosocial prevention programme that helps people with recurrent depression stay well in the long term. It was singled out in the 2009 National Institute for Health and Clinical Excellence (NICE) Depression Guideline as a key priority for implementation. Despite good evidence and guideline recommendations, its roll-out and accessibility across the UK appears to be limited and inequitably distributed. The study aims to describe the current state of MBCT accessibility and implementation across the UK, develop an explanatory framework of what is hindering and facilitating its progress in different areas, and develop an Implementation Plan and related resources to promote better and more equitable availability and use of MBCT within the UK National Health Service.NIHRHS&D